Contributes to #5061

This repo contains a few tests on LightGBM's C API which are run with pytest.

Today, they load lib_lightgbm.{dll/so} using a modified copy of the same library-loading code with ctypes used in the Python package. LightGBM's CI runs python setup.py bdist_wheel, then these C API tests rely on a lib_lightgbm.{dll/so} being available in python-package/compile/.

This PR proposes removing that reliance on the repo structure + exactly where lib_lightgbm.so copies get left behind after building Python wheels.

How does this contribute to #5061? The changes for that (in-progress in #5759) will involve building lightgbm wheels in an isolated build directory, which breaks the assumptions in c_api_test/test_.py that a lib_lightgbm will be left behind in the python-package/ directory.

How I tested this

Check that this works with the Python package installed.

pip uninstall --yes lightgbm
cd python-package/
python setup.py bdist_wheel
pip install dist/*.whl
cd ..
pytest tests/c_api_test

Check that it works without the Python package.

pip uninstall --yes lightgbm
rm -rf ./python-package/build
rm -rf ./python-package/compile
rm -rf ./python-package/dist
mkdir build
cd build
cmake ..
make -j3
cd ..
pytest tests/c_api_test